Add quiz report visibility control for coaches #13064

LianaHarris360 · 2025-02-11T17:32:26Z

Summary

This pull request implements the functionality to enable coaches to select when a quiz report is visible to learners. Coaches can now choose to allow learners to see their quiz report immediately after submitting the quiz or only after the coach ends the quiz.

The Exam model, api, and serializer have been updated to include the boolean field instant_report_visibility, which defaults to true, indicating that learners will instantly see their quiz report once they complete a quiz. When this field is false, learners will see their quiz reports after the quiz is closed. A new migration file has been created to include the new field to the exam model.

When creating a new quiz, the icon circleCheckmark beside the Report Visibility selection button will be available in KDS V5.0.0

Creating a new quiz:

CreateNewQuiz.mov

Learner quiz report unavailable:

CannotViewQuizReport.mov

Quiz Status page:

References

Figma (see "Quiz report Visibility" including "Learner Experience" below)
Closes #12902

Reviewer guidance

Create a quiz as a coach, and select 'After learner submits quiz' or 'After coach ends the quiz'.
Take the quiz as a learner and confirm that the quiz report is shown or not shown to the learner, based on the quiz instant_report_visibility
Confirm that the learner can see the quiz report if the quiz has been closed.
After the coach ends the quiz, the Report Visibility section on the Quiz Status page is not displayed.

…d to allow coach selection of report visibility

…ction and descriptions

…r instant_report_visibility is true

github-actions · 2025-02-11T18:07:16Z

Build Artifacts

Asset type	Download link
PEX file	kolibri-0.18.0a1.dev0_git.74.g484d1375.pex
Windows Installer (EXE)	kolibri-0.18.0a1.dev0+git.74.g484d1375-windows-setup-unsigned.exe
Debian Package	kolibri_0.18.0a1.dev0+git.74.g484d1375-0ubuntu1_all.deb
Mac Installer (DMG)	kolibri-0.18.0a1.dev0+git.74.g484d1375.dmg
Android Package (APK)	kolibri-0.18.0a1.dev0+git.74.g484d1375-0.1.4-debug.apk
Raspberry Pi Image	kolibri-pi-image-0.18.0a1.dev0+git.74.g484d1375.zip
TAR file	kolibri-0.18.0a1.dev0+git.74.g484d1375.tar.gz
WHL file	kolibri-0.18.0a1.dev0+git.74.g484d1375-py2.py3-none-any.whl

nucleogenesis

I gave the code a look and had one minor question. I think that so long as the exam's value for the visibility field reflects the exam's previously saved value then it's all good.

Overall, the code changes here LGTM! I'll leave approval to QA and others.

nucleogenesis · 2025-02-12T16:33:57Z

kolibri/plugins/coach/assets/src/views/common/assignments/AssignmentDetailsModal.vue

@@ -217,6 +256,7 @@
        formIsSubmitted: false,
        showServerError: false,
        showTitleError: false,
+        instantReportVisibility: true,


Should this be dynamic rather than hard-set? For example, when I open an exam for editing, should this be set to the value previously saved to the exam?

Oi, just realized that you've set it in mounted - curious if you tried assigning it to instantReportVisibility: this.assignment.instant_report_visibility || true; here rather than in mounted below?

Hmm, the only thing is if this.assignement.instant_report_visibility is false, true would still be returned. But I could do:
instantReportVisibility: this.assignment.instant_report_visibility ?? true
Since doing so would remove the need to set it in the mounted hook, I will update the PR.

I've just read that the Nullish coalescing operator (??) is mainly supported on newer browsers so I've decided to set it as instantReportVisibility: this.assignment.instant_report_visibility !== false, because false !== false evaluates to false, and for all other values, it is set to true.

Update: I went with setting instantReportVisibility: this.assignment.instant_report_visibility, since that is what previously happened in the mounted hook anyway.

Setting it as: instantReportVisibility: this.assignment.instant_report_visibility !== false, caused lessons to also contain a instant_report_visibility which broke the tests, and only quizzes should have this field.

This field is set to true by default in kolibri/plugins/coach/assets/src/composables/quizCreationSpecs.js, so when a new quiz is created, it has the correct setting. So it was unnecessary to set it in mounted(), thanks for pointing this out!

…_visibility field

…bility field

marcellamaki · 2025-02-18T16:00:44Z

@radinamatic @pcenov - could this go on the QA team list for tomorrow? Thank you!

pcenov · 2025-02-19T15:36:30Z

Hi @LianaHarris360 - I confirm that everything is implemented and working as specified above.
The only issue that I want to mention is that when I have assigned a quiz to a learner on a learn only device, then if the learner is at the page with the assigned quizzes just after the sync with the server has happened the learner can click on the quiz card which is displayed as if the report can be previewed while for some reason it has not loaded yet. If I navigate to another plugin and come back then everything is fine. This is an edge case but can be confusing so if it can be handled better that would be great!

LOD.mp4

nucleogenesis · 2025-02-19T22:13:39Z

IMO the issue @pcenov reported re: the synced data not being reflected in the Learn UI might be an issue beyond the scope of this PR and should probably be a new issue addressed separately - should be confirmed w/ @marcellamaki

radinamatic · 2025-02-20T01:39:50Z

Similar to recent PR I tested for non-editable quizzes from previous versions, it seemed to me that this feature could also have potential issues during upgrade scenarios, so I upgraded a 0.16.2 Kolibri install with the asset from this PR.

Class on that 0.16.2 facility that was upgraded had 2 quizzes that were not ended by the coach, but the learner had already submitted one of them (and presumably already seen the report).
- If the coach goes and edits the quiz details to change the report visibility to after the quiz ends, learner is not able to open the report anymore and modal appears as expected.
- However, once coach ends the quiz, that same learner is still unable to open the report, and they continue to see the modal. No amount of navigating away and back on either learner or coach side makes it possible for learner to see the report again.
  
  reports-visible.mp4
  
  Granted this seems like an edge case, but just raises some concerns and doubts whether we considered all the implications that this new report visibility feature can have in a variety of upgrade scenarios
Another thing I noticed (marked at timestamp 1:44), is that report on the coach side states that this same quiz ended 1 minute ago, but the report was made visible 207 days ago... (F)actually correct, but very confusing!

Full home folder used in this test case.

rtibbles · 2025-02-20T02:30:11Z

kolibri/core/exams/models.py

@@ -158,6 +158,10 @@ class Meta:
    """
    data_model_version = models.SmallIntegerField(default=3)

+    # If True, learners have instant access to exam reports after submission.
+    # Otherwise, reports are visible only after the coach ends the exam.
+    instant_report_visibility = models.BooleanField(default=True)


I think we may want to be careful about exactly how we add this field - my specific thought is in relation to the migration this will produce on KDP.

When we add a Django model field with a default value, this can result in a very long running migration, if the table is large.

The way to avoid this is to allow the field to be nullable null=True - and handle the null values, and return them as True via the API.

We create the migration with null=True and no default. Then we add the default=True and create another migration. At this point, we can merge the two migration files into one, just as long as the AddField operations are first, and then the AlterField to add the default comes after.

Yes, this is a good point. It would be simpler and safer to allow the field to be null and return a True value for it. I will update the field and do the migration adjustments.

LianaHarris360 · 2025-02-20T15:47:02Z

Thank you for adding the home folder for that use case @radinamatic I think navigating away from the plugin and back to see the report is a problem I can fix within this PR. The learner should be able to see those updates as soon as they're available, especially if they’ve just recently synced. However, I can see how this might be outside the scope of the PR, so I can open an separate issue to address this if necessary.

And the report visibility timestamp that the coach sees after the ending the quiz should be updated to reflect the correct timestamp that the report was available, if it was set to “After coach ends the quiz”. I will update this pull request to fix that.

…t=True and merge migration changes

…nt_report_visibility is false

rtibbles · 2025-02-24T21:04:08Z

kolibri/core/exams/api.py

@@ -69,6 +69,7 @@ class ExamViewset(ValuesViewset):
        "creator",
        "data_model_version",
        "learners_see_fixed_order",
+        "instant_report_visibility",


The only thing I can think of here, is that we should probably return True for this if the backend value is null - that will save the frontend from having to worry about the nullability of the field, and leave it entirely as a backend concern (as we only really introduced it for the purposes of migration).

LianaHarris360 added 7 commits February 11, 2025 08:47

Update exam model, views, and api with instant_report_visibility fiel…

09301e8

…d to allow coach selection of report visibility

Add exam field instant_report_visibility for use in the frontend

3b64d66

Update commonCoachStrings with new strings for report visibility sele…

e39ccdb

…ction and descriptions

Add report visibility selection button to frontend quiz creation

39bb753

Add report visibility label & description to quiz status page

fa607e3

Update learner view to show quiz reports only if the exam is closed o…

972728c

…r instant_report_visibility is true

Update exam api tests with instant_report_visibility field

272a948

LianaHarris360 added this to the Kolibri 0.18: Coach Resource Discovery milestone Feb 11, 2025

LianaHarris360 requested review from rtibbles, nucleogenesis and marcellamaki February 11, 2025 17:32

github-actions bot added DEV: backend Python, databases, networking, filesystem... APP: Learn Re: Learn App (content, quizzes, lessons, etc.) APP: Coach Re: Coach App (lessons, quizzes, groups, reports, etc.) DEV: frontend labels Feb 11, 2025

nucleogenesis reviewed Feb 12, 2025

View reviewed changes

LianaHarris360 added 2 commits February 12, 2025 12:15

Set data prop instantReportVisibility using assignment.instant_report…

80645ac

…_visibility field

Set instantReportVisibility to default assignment instant_report_visi…

41f8f75

…bility field

marcellamaki mentioned this pull request Feb 18, 2025

Unstarted quiz page opened from learner summary opens empty and is unclear #13057

Open

rtibbles reviewed Feb 20, 2025

View reviewed changes

LianaHarris360 added 3 commits February 20, 2025 12:02

Update exam instant_report_visibility field with null=True and defaul…

39a71e1

…t=True and merge migration changes

Display the report visibility timestamp as exam archive date if insta…

d026a02

…nt_report_visibility is false

Add updated migration field operations for exams

c4fca87

rtibbles reviewed Feb 24, 2025

View reviewed changes

Return true for null instant_report_visibility values

995180c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add quiz report visibility control for coaches #13064

Add quiz report visibility control for coaches #13064

LianaHarris360 commented Feb 11, 2025

github-actions bot commented Feb 11, 2025 •

edited

Loading

nucleogenesis left a comment

nucleogenesis Feb 12, 2025

nucleogenesis Feb 12, 2025

LianaHarris360 Feb 12, 2025 •

edited

Loading

LianaHarris360 Feb 12, 2025

LianaHarris360 Feb 13, 2025

marcellamaki commented Feb 18, 2025

pcenov commented Feb 19, 2025

nucleogenesis commented Feb 19, 2025

radinamatic commented Feb 20, 2025

rtibbles Feb 20, 2025

LianaHarris360 Feb 20, 2025

LianaHarris360 commented Feb 20, 2025

rtibbles Feb 24, 2025

Add quiz report visibility control for coaches #13064

Are you sure you want to change the base?

Add quiz report visibility control for coaches #13064

Conversation

LianaHarris360 commented Feb 11, 2025

Summary

References

Reviewer guidance

github-actions bot commented Feb 11, 2025 • edited Loading

nucleogenesis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LianaHarris360 Feb 12, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcellamaki commented Feb 18, 2025

pcenov commented Feb 19, 2025

nucleogenesis commented Feb 19, 2025

radinamatic commented Feb 20, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LianaHarris360 commented Feb 20, 2025

Choose a reason for hiding this comment

github-actions bot commented Feb 11, 2025 •

edited

Loading

LianaHarris360 Feb 12, 2025 •

edited

Loading